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ABSTRACT 


1 


This  report  describes  an  analysis  of  thirteen  Advanced 
Scout  Helicopter  (ASH)  candidates  and  mixtures  of  those 
candidates. 

The  analysis  evaluates  the  candidates  on  the  basis  of 
their  military  worth;  life  cycle  costs;  attainability;  force 
structure  personnel  impact;  and  rationalization ,  standardiza¬ 
tion,  and  interoperability  impact.  These  major  evaluation 
categories  are  subdivided  in  such  a  way  that  over  seventy 
attributes  of  value  are  considered. 

The  analysis  identifies  as  the  best  ASH  candidate  the 
new  development  with  a  single  advanced  technology  engine. 

In  addition,  three  candidates  and  one  mixture  of  candidates 
are  identified  as  the  best,  according  to  the  criteria  specified 
above,  if  cost  is  constrained  at  lower  levels. 

The  major  portion  of  the  report  describes  and  explains 
the  methodology  used  to  evaluate  the  candidates,  presents 
the  results  of  the  analysis,  and  illustrates  several  of  the 
sensitivity  analyses  that  were  performed.  Detailed  assess¬ 
ments  used  in  the  analysis  and  extensive  rationale  supporting 
those  assessments  are  reported  in  the  appendices. 


The  report  also  describes  an  analysis  aimed  at  improving 
the  design  of  ASH  candidates.  Because  this  analysis  was  not 
developed  beyond  an  early  stage,  its  results  should  not  be 
used  to  draw  conclusions  about  any  ASH  design.  r 
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1.0  INTRODUCTION 


1. 1  Background 

The  Advanced  Scout  Helicopter  Special  Study  Group 
(ASH  SSG)  has  been  tasked  by  the  Army  with  defining  and 
demonstrating  the  need  for  an  ASH,  with  selecting  an  effec¬ 
tive  and  affordable  ASH  program,  and  with  demonstrating  that 
the  chosen  ASH  alternative  is  more  cost  effective  than  the 
other  alternatives  considered.  The  ASH  SSG  must  build 
support  for  the  ASH  based  upon  an  understanding  of  basic 
operational  and  organizational  concepts.  Their  studies  must 
be  requirements  rather  than  hardware  oriented.  In  particular, 
the  ASH  SSG  must  demonstrate  that  the  ASH  need  cannot  simply 
be  met  by  the  current  Light  Observation  Helicopter  (LOH) 
being  used  for  the  ASH  role. 

1 . 2  Objectives 

Given  the  above  described  need,  Decisions  and  Designs, 
Inc.  (DDI)  set  out  to  assist  the  ASH  SSG  by  constructing  a 
comprehensive  ASH  evaluation  model  utilizing  multi-attribute 
utility  assessment  (MAUA)  modeling.  MAUA  is  a  form  of 
decision-analytic  modeling  that  allows  the  incorporation  of 
all  objective  as  well  as  subjective  data  that  might  influ¬ 
ence  the  choice  of  one  ASH  candidate  over  another.  Both 
hardware  and  operational  characteristics  of  the  candidates 
are  incorporated  in  this  single  model.  Simulation  data  and 
results  are  included  as  well.  The  output  of  the  MAUA  model 
is  a  numerical  representation  of  the  worth  of  each  ASH 
candidate.  These  numbers  are  highly  supportable  via  care¬ 
fully  recorded  written  rationale  and,  together  with  the 
relevant  cost  data,  can  be  used  to  calculate  the  cost  effect¬ 
iveness  of  each  ASH  candidate.  In  the  remainder  of  this 
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report,  the  nature  of  the  entire  modeling  effort  for  the  ASH 
SSG  is  thoroughly  detailed. 


1.3  Descriptions  of  ASH  Candidates 

The  main  portion  of  this  project  involved  an  evaluation 
of  thirteen  candidate  Advanced  Scout  Helicopter  (ASH)  designs 
and  mixes  of  these  designs.  These  candidates  are  listed, 
together  with  three-character  identifiers,  in  Table  1-1. 
Detailed  descriptions  of  the  candidates  are  given  in  the 
"Special  ASH  Issue"  of  Army  Aviation*  and  will  not  be 
repeated  here.  We  will,  however,  point  out  some  of  the 
major  characteristics  of  the  candidates  in  paragraphs  below. 

The  first  four  candidates  listed  are  completely  new 
developments.  These  candidates  are  the  result  of  design 
studies  that  started  from  scratch  with  a  "clean  sheet  of 
paper,"  unconstrained  by  current  helicopter  designs.  All  of 
these  designs  incorporate  a  full  complement  of  mission 
equipment;  they  differ  mainly  in  the  number  and  power  of 
engines — although  all  designs  incorporate  an  Advanced  Tech¬ 
nology  Engine  (ATE) — and  in  the  seating  arrangement  for  the 
crew.  BTA  is  a  single-engine  helicopter  that  configures  the 
crew  in  a  side-by-side  seating  arrangement.  BT2  has  twin 
engines  and  a  side-by-side  configuration.  BTT  has  twin 
engines  and  a  tandem  configuration  (with  a  frag  barrier  that 
provides  the  crew  with  protection  against  a  23  millimeter 
high  explosive  round) .  B4K  has  twin  engines  and  a  tandem 
configuration;  it  also  has  the  capability  to  operate  under 
4,000  feet/95°  F  conditions  with  one  engine  inoperative. 

The  next  six  candidates  are  modifications  of  existing 
attack  helicopters.  The  first  three  candidates  are  varia¬ 
tions  of  the  OH-58Cc  The  OH-58D  is  a  minimal  modification 


1  Volume  28,  Number  10,  1  October  1979. 


1 


New  Developments 


BTA: 

New 

Dev 

1 

X 

ATE 

BT2: 

New 

Dev 

2 

X 

ATE, 

sxs 

BTT: 

New 

Dev 

2 

X 

ATE, 

Tandem 

B4X: 

New 

Dev 

2 

X 

ATE, 

SXS,  with  4K/95  OEI 

Modifications 

58C:  OH-58C 

58D:  OH-58D 

58E:  OH-58E 

OHT:  OH-1  with  TADS 

OHM:  OH-1  with  MMS 

064:  OH-64 

Others 

'  350:  Aerospatiale  AS  350 
129:  Agusta  A129 
500:  Hughes  500D 


Table  1-1 

THE  THIRTEEN  ASH  CANDIDATES 


3 


that  involves  principally  the  addition  of  a  mast-mounted 
sight  (MMS) ,  with  day-only  capability,  and  some  extra  mis¬ 
sion  equipment  (the  OH-58D  does  not  meet  the  specified  ASH 
maneuverability  requirement) .  The  OH-58E  is  a  more  sub¬ 
stantial  modification  that  includes  a  four-bladed  rotor,  a 
MMS  equipped  with  day-television  and  forward-looking  infrared 
(FLIR) — for  night  operations--and  an  upgraded  transmission 
and  engine.  The  next  two  candidates  are  modifications  of 
the  AH-1  attack  helicopter.  OHT  is  the  minimal  modification 
that  fits  the  AH-1  airframe  with  a  nose-mounted  target 
acquisition  and  designation  system  (TADS) .  OHM  is  a  more 
substantial  modification  that  fits  the  AH-1  airframe  with  a 
four-bladed  rotor  and  the  full  complement  of  ASH  mission 
equipment  with  a  modular  MMS.  The  OH-64  is  a  modification 
of  the  YAH- 6 4  attack  helicopter  that  leaves  the  weapons 
systems  intact  but  removes  the  weapons . 

The  final  three  candidates  are  derivatives  of  heli¬ 
copters  that  are  currently  in  existence  or  under  develop¬ 
ment.  These  include  the  Aerospatial  AS-350  (350),  the 
Agusta  A-129  (129) ,  and  the  Hughes  500D  (500) . 


2.0  THE  EVALUATION  MODEL 


This  chapter  describes  the  results  of  an  effort  to 
capture  in  a  single  comprehensive  model  the  many  factors 
that  could  lead  the  Army  to  prefer  one  ASH  candidate  over 
another.  Thus,  it  represents  a  substantial  extension  beyond 
the  scope  of  those  studies  which  deal  only  with  cost,  or 
only  with  operational  effectiveness.  The  purpose  of  a 
comprehensive  model  such  as  the  one  presented  here  is  to 
provide  a  single  yardstick  to  compare  systems  by  trading  off 
one  criterion  against  another  according  to  their  relative 
importance. 

2. 1  General  Description  of  the  Methodology 

2.1.1  Overview  -  The' evaluation  model  is  an  instance 
of  a  methodology  called  Multi-Attribute  Utility  Analysis 
(MAUA) .  In  general,  MAUA  is  characterized  by  the  represen¬ 
tation  of  outcomes  in  terms  of  a  number  of  different  cri¬ 
teria,  rather  than  a  single  global  measure  of  value.  Various 
forms  of  MAUA  are  possible,  differing  in  terms  of  the  specific 
method  of  representing  an  option's  score  on  each  criterion, 
the  structure  of  the  relations  that  are  assumed  to  hold 
among  the  criteria,  and  the  method  of  aggregating  the  com¬ 
ponent  ratings  into  a  single  overall  evaluation  for  each 
option. 


The  evaluation  model  can  be  characterized  as  a 
relative,  hierarchical,  or  weighted  additive  utility  model. 
Each  of  these  three  concepts  is  explained  below. 

2. 1.1.1  Relative  scoring  -  In  a  relative  model, 
scores  on  each  of  the  attributes  represent  not  the  propor¬ 
tional  values  of  the  various  options,  but  rather  the  relative 
differences  among  the  options.  The  distinction  between 
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relative  and  absolute  scoring  is  critical  to  a  proper  inter¬ 
pretation  of  MAUA  results,  so  amplification  is  in  order:  An 
"absolute"  scoring  system  necessitates  defining  a  true  zero 
level  of  performance  and  scoring  systems  proportional  to  how 
far  they  exceed  that  zero  level;  a  "relative"  scoring  system 
arbitrarily  selects  the  least  desirable  outcome  on  each 
criterion  as  a  relative  zero,  and  then  scores  each  of  the 
other  systems  proportional  to  the  magnitude  of  the  difference 
between  that  system  and  the  one  with  the  lowest  score. 

As  illustrated  in  Section  2.1.2, 
either  type  of  scale  may  be  used  to  evaluate  options.  It 
turns  out,  however,  that  very  often  the  "true  zero"  on  many 
important  attributes  is  so  much  worse  than  the  options 
actually  being  considered  that  it  is  difficult  to  define  or 
to  think  of.  Worse  still,  there  is  often  no  obvious  notion 
of  a  zero  level  of  importance  with  any  meaning  to  the  deci¬ 
sion  makers.  Finally,  even  if  these  problems  do  not  arise, 
the  numerical  ratios  may  be  so  close  as  to  appear  indistin¬ 
guishable,  even  when  the  magnitudes  of  the  differences  are 
quite  large.  For  the  above  reasons,  it  is  often  preferable 
to  use  relative  scoring,  which  restricts  its  domain  of 
attention  to  the  actual  range  of  variation  among  the  options. 

The  benefits  of  relative  scoring  are 
that  it  spreads  the  numerical  scores  out  for  better  discrimi¬ 
nation  among  the  options,  that  it  involves  considering  only 
the  realistic  options  being  evaluated  (rather  than  hypothe¬ 
tical  constructs  sue'  as  the  absolute  zero) ,  and  that  it 
requires  as  few  elicited  values  as  possible  to  arrive  at  a 
meaningful  set  of  scores.  Disadvantages  include  an  inability 
to  determine  whether  a  given  system  is  "bad"  or  "good"  (only 
"better"  or  "worse"  are  meaningful) ,  the  need  for  caution  in 
interpreting  the  numerical  scores,  and  the  impossibility  of 
determining  whether  the  "best"  system  is  really  significantly 
better  than  any  of  the  others.  Overall,  if  the  goal  is  to 


6 


select  the  best  system,  the  advantages  of  relative  scoring 
outweigh  its  problems  (so  long  as  proper  caution  is  exer¬ 
cised  in  interpreting  numerical  results) . 

2. 1.1. 2  Hierarchical  utility  structure  -  It 
would  be  possible  to  list  the  entire  set  of  attributes  that 
might  affect  the  preference  for  one  alternative  over  another; 
but  such  a  method  would  be  highly  cumbersome,  difficult  to 
communicate,  and  susceptible  to  a  number  of  methodological 
biases  that  result  from  the  overload  of  processing  large 
amounts  of  information  without  subdividing  it  into  manage¬ 
able  "chunks."  One  solution,  which  is  adopted  here  and  used 
in  a  variety  of  similar  analyses,  is  to  develop  a  hierarchical 
structure  which  expresses  the  overall  value  as  the  aggregate 
of  a  small  number  of  major  attributes;  each  of  these  major 
attributes  may  itself  be  subdivided  ("decomposed")  into 
minor  attributes,  which  may  themselves  be  decomposed,  et  cetera. 

A  hierarchical  utility  structure  may  be 
represented  as  a  schematic  tree  in  which  the  overall  value 
appears  at  the  top  level;  each  of  the  major  attributes 
appears  as  a  branch  beneath  that  top  level;  and  component 
sub- attributes  of  a  major  attribute  appear  as  branches 
beneath  it.  Thus,  by  referring  to  any  label  on  the  tree, 
one  can  simultaneously  indicate  not  only  the  specific  attri¬ 
bute  named,  but  everything  which  appears  beneath  it  (connected 
by  branches)  as  well.  (The  hierarchical  branching  structure 
of  the  ASH  decision  model  is  displayed  in  Section  2.2.1.) 

The  process  of  hierarchical  decomposition 
provides  three  important  benefits:  first,  it  breaks  the 
elicitation  process  up  into  "chunks"  of  manageable  size; 
second,  it  organizes  the  presentation  of  final  results, 
highlighting  the  most  important  factors  without  losing  the 
ability  to  retrieve  details  when  desired;  and  third,  it 
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limits  the  required  assessments  to  comparisons  among  at¬ 
tributes  that  are  closely  related  in  meaning,  and  therefore 
relatively  easy  to  weigh  against  one  another. 


2.1.1. 3  Weighted  additive  utility  aggregation  - 
When  a  given  utility  attribute  (including  the  "overall 
value")  has  been  subdivided  into  components,  various  rules 
may  be  used  to  combine  the  component  scores  into  a  single 
summary  score.  Most  well-structured  hierarchical  utility 
models  are  best  treated  by  a  weighted  additive  aggregation 
rule:  the  summary  score  on  a  higher-level  attribute  is 
equal  to  the  sum  (or,  more  properly,  the  average)  of  the 
component  scores,  each  weighted  according  to  its  assessed 
impact  on  the  final  value.  These  weights  are,  roughly 
speaking,  a  combination  of  relevance,  importance,  and  varia¬ 
tion  among  the  options  with  respect  to  each  attribute; 
weights  are  arbitrarily  re-scaled  in  proportion  to  one 
another  so  that  the  adjusted  or  "normalized"  values  add  to 
one  (or  to  100%) . 


For  example,  if  the  weights  of  three 
attributes  are  .40,  .25,  and  .35,  and  if  a  given  option's 
scores  on  those  attributes  are  80,  100,  and  40,  respectively, 
the  overall  score  for  that  option  will  be  (80  x  .40)  +  (100 
x  .25)  +  (40  x  .35)  =  71.  In  order  to  interpret  the  value 
thus  obtained,  assuming  a  relative  scoring  system,  the  user 
must  consider  two  (probably  hypothetical)  options:  one 
which  would  combine  all  the  most  desirable  features  of  the 
various  options,  resulting  in  a  score  of  100  on  every  attri¬ 
bute  (and  therefore  in  an  overall  score  of  100) ;  and  another 
which  would  combine  all  the  worst  aspects  of  the  various 
options,  resulting  in  a  (relative)  score  of  zero  throughout. 
In  a  relative  model,  a  score  of  71  would  mean  that  the  given 
option  was  much  closer  to  the  100-point  "best”  than  to  the 
0-point  "baseline"  option.  (Again,  the  user  must  avoid 
making  unjustified  claims  about  the  absolute  value  of  an 


option,  or  about  the  ratio  of  the  scores  of  two  options;  the 
only  valid  conclusions  involve  the  rank  order  of  preference 
among  the  options  and  the  relative  magnitudes  of  the  differ¬ 
ences  when  compared  with  one  another.) 

One  difficulty  inherent  in  relative 
modeling  is  the  definition  and  interpretation  of  the  weights 
assigned  to  various  attributes.  In  any  additive  model,  the 
ultimate  criterion  for  interpreting  and  defining  weights  is 
the  following:  if  an  increase  of  p  points  on  Attribute  X  is 
valued  exactly  as  highly  as  an  increase  of  q  points  on  Attri¬ 
bute  Y,  everything  else  remaining  equal,  then  the  two  attri¬ 
butes  should  be  assigned  weights  in  the  proportion  q:p 
(i.e.,  the  more  points  needed  on  Attribute  X  to  match  the 
effect  of  the  other  attribute,  the  less  weight  Attribute  X 
deserves) . 

Again,  because  of  the  pitfalls  in 
interpreting  relative  scores,  caution  must  be  taken  not  to 
identify  the  weight  on  a  given  attribute  as  the  "importance" 
of  that  attribute;  a  critically  important  attribute  on  which 
the  options  are  all  identical  will  properly  receive  a  weight 
of  zero.  On  the  other  hand,  if  the  performances  of  the  op¬ 
tions  on  all  of  the  most  important  attributes  are  very 
close,  a  seemingly  low-priority  item  on  which  there  is 
substantial  variation  may  in  fact  receive  the  largest  weight. 

2. 1.1. 4  Summary  of  modeling  procedure  -  The 
entire  procedure  of  MAUA,  as  used  in  the  evaluation  model 
can  be  divided  into  the  following  sequence  of  steps: 

Step  1  -  define  the  options  to  be  evaluated; 

Step  2  -  define  the  attributes  which  will  contrib¬ 
ute  to  overall  utility,  and  the  hier¬ 
archical  structure  by  which  they  are 
organized; 


Step  3  -  score  each  of  the  "data- level"  attributes 

(i.e.,  those  which  are  not  further  subdivided) 
by  assigning  numerical  values  to  each  option 
on  a  0-to-100-point  relative  scale; 

Step  4  -  for  every  attribute  which  is  subdivided  into 
components,  assess  the  weights  of  its  sub¬ 
attributes; 

Step  5  -  starting  with  the  "data-level"  attributes  and 
progressing  up  the  tree,  calculate  the  summary 
score  of  each  option  as  a  weighted  sum  of  its 
component  scores  on  the  sub-attributes,  so 
that  the  summary  scores  at  the  "overall"  level 
represent  the  evaluations  of  the  options. 

Having  completed  these  steps,  the 
analysis  team  proceeds  to  explicate  the  results,  perform 
sensitivity  analysis,  and  otherwise  facilitate  the  communica¬ 
tion  of  the  study's  implications.  Generally,  most  of  the 
effort  in  a  good  analysis  is  spent  in  these  latter  activities. 

2.1.2  A  comparison  of  three  methods  of  scaling  -  In  the 
process  of  structuring  the  ASH  evaluation  model,  a  question 
arose  over  the  best  method  of  scaling  to  use.  In  particular, 
some  concern  was  expressed  that  the  relative  scales,  which 
measure  the  value  of  the  ASH  candidates  on  the  0-to-100  scale 
explained  above,  unjustly  penalize  the  worst  candidates  of  the 
group.  An  argument  was  made  that  the  use  of  an  absolute 
scale  or  a  ratio  scale  might  avoid  this  problem  and  thus 
might  be  a  better  approach.  The  following  discussion  high¬ 
lights  the  similarities  and  differences  among  these  three 
methods  of  evaluation  and  illustrates  how  all  methods  yield 
equivalent  results  when  each  is  done  properly. 
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The  three  methods  of  scaling  value  will  be 
illustrated  by  using  the  following  example.  Suppose  that  a 
decision  maker  is  interested  in  choosing  the  best  helicopter 
from  among  the  following  set  of  three: 

(a)  a  hypothetical  new  development; 

(b)  a  hypothetical  modification  of  an  existing 
helicopter; 

(c)  a  hypothetical  existing  helicopter. 

Suppose  further  that  the  decision  maker  wishes  to  make  this 
choice  based  on  three  criteria:  procurement  cost,  military 
effectiveness,  and  reliability. 

2. 1.2.1  The  absolute  scale  -  The  first  step  in 
an  evaluation  is  to  determine  a  score  for  each  alternative 
on  each  criterion.  This  procedure  is  relatively  simple  for 
procurement  cost.  The  decision  maker  specifies  his  utility 
for  money  over  a  relevant  range  and  estimates  the  cost  for 
each  alternative.  There  is,  however,  already  a  problem  with 
an  absolute  scale;  the  relevant  range  must  be  defined.  To 
be  truly  absolute,  the  scale  might  have  to  extend  at  least 
from  a  cost  of  zero  to  a  very  large  number  that  might  approach 
positive  infinity.  To  be  useful,  though,  the  decision  maker 
must  define  some  smaller  range  that  encompasses  the  entire 
range  over  which  he  can  differentiate  a  value  for  reducing 
cost.  Assume  that  the  decision  maker  defines  this  "absolute" 
range  as  $0  to  $5  million  per  helicopter  and  that  his 
preference  is  linear  with  cost.  We  can  then,  without  loss 
of  generality,  scale  the  decision  maker's  preference  for 
cost  to  be  in  the  interval  0  to  100  using  the  function: 

U(C)  »  100  ($5  million  -  C) ,  where  C  is  cost  in  dollars. 


Suppose  that  the  following  are  the  estimated  costs  for  the 
alternatives/  which  can  be  converted  into  preference  for 
cost  by  using  the  above  transformation: 


Helicopter 


Cost 


Absolute  Score 
for  Cost 


A  $4.5  million 
B  $4  million 
C  $2  million 


10 

20 

60 


Next,  consider  the  criterion  of 
military  effectiveness.  For  this  criterion,  the  decision 
maker  has  another  problem:  to  establish  operational  defini¬ 
tions  of  "perfect"  and  "unacceptable"  military  effectiveness 
in  absolute  terms — or  to  use  a  more  measurable  quantity  as  a 
surrogate.  For  our  example,  assume  that  the  decision  maker 
uses  Specific  Exchange  Ratio  (SER) ,  as  estimated  in  a  Simula 
tion,  as  a  surrogate  for  military  effectiveness.  Again, 
assume  that  preference  is  linear,  and  further  assume  that 
the  decision  maker  judges  an  SER  of  0  to  be  totally  unaccept 
able  and  one  of  40  to  be  perfect.  The  alternatives  might 


be  scored  as 

follows : 

Absolute 

Helicopter 

SER 

Score 

A 

30 

75 

B 

25 

60 

C 

15 

37.5 

Now,  consider  the  third  criterion, 
reliability.  Assume  that  the  decision  maker  uses  Mainte¬ 
nance  Man-Hours  per  Flight  Hour  (MMH/FH)  as  a  surrogate  for 
reliability;  a  value  of  0  MMH/FH  is  considered  perfect,  a 
value  of  5  MMH/FH  is  considered  acceptable,  and  preference 
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is  linear  between  these  extremes.  The  alternatives  might 
then  be  scored  as  follows: 


Helicopter 

A 

B 

C 


MMH/FH 

3 

4 
2 


Absolute 

Score 


40 

20 

60 


The  relationships  between  the  absolute  criteria  and  their 
corresponding  utility  functions  are  shown  in  Figure  2-1. 

2. 1.2. 2  The  ratio  scale  -  The  decision  maker 
could  avoid  the  problem  of  defining  a  perfect  capability,  in 
the  absolute  sense,  by  identifying  the  best  alternative  on 
each  criterion  and  measuring  the  performance  of  the  other 
alternatives  as  a  ratio  to  the  best  one.  Consider,  for 
instance,  the  criterion  of  military  effectiveness.  Here, 
helicopter  A  is  best,  helicopter  C  is  half  as  good  as  A,  and 
helicopter  B  is  80%  as  good  as  A.  So,  scores  on  a  ratio 
scale  would  be  as  follows: 


Ratio 


Helicopter 

SER 

Score 

A 

30 

100 

B 

25 

80 

C 

15 

50 

Similar  transformations  are  possible 
with  the  other  scales  as  well,  but  with  the  other  scales  the 
decision  maker  must  be  careful  with  this  transformation.  In 
particular,  for  a  ratio  scale  to  be  compatible  with  the 
absolute  scale,  the  "unacceptable"  point  of  zero  value  must 
remain  the  same.  Consider  the  criterion  of  procurement 
cost.  Here,  helicopter  A  is  best  relative  to  the  unacceptable 
cost  of  $5  million  per  unit,  helicopter  B  is  one-third  as 
good  as  C  relative  to  the  unacceptable  cost  *  1/3, 
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(Military  Effectiveness) 
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(Reliability) 


Figure  2-1 

ABSOLUTE  VALUE  SCALES 
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and  helicopter  A  is  one- sixth  as  good  as  C.  These  observa 
tions  lead  to  the  following  ratio  scale  for  cost: 


Ratio  Score 
(assuming  $5  million 

Helicopter  Cost  is  unacceptable) 


A  $4.5  million 
B  $4.0  million 
C  $2.0  million 


16.7 

33.3 

100.0 


Using  a  similar  argument  gives  the  following  ratio  scale  for 
reliability  (relative  to  an  unacceptable  level  of  5  MMH/FH) : 


Helicopter 


MMH/FH 


Ratio  Score 


A 

B 

C 


100.0 

33.3 

66.7 


2. 1.2. 3  The  relative  scale  -  Notice  from  above 
that  both  the  absolute  and  ratio  scales  force  the  decision 
maker  to  define  an  unacceptable  level  and  an  absolute  zero 
on  each  scale.  A  completely  relative  scale,  however,  does 
not  force  such  definitions. 

Here,  the  scales  were  defined  by  the 
alternatives  under  consideration.  With  the  best  helicopter 
on  the  criterion  defining  the  score  of  100,  the  worst  heli¬ 
copter  defining  the  score  of  0,  and  the  other  helicopter 
being  scored  in  relation  to  the  other  two,  the  following 


ive  scales 

are  defined: 

Helicopter 

Cost 

Relative  Score 

A 

$4.5  million 

0 

B 

$4.0  million 

20 

C 

$2.0  million 

100 

15 


SER 

A 

30 

100 

B 

25 

60 

C 

15 

0 

Helicopter 

MMH/FH 

Relative  Score 

A 

3 

50 

B 

4 

0 

C 

2 

100 

Relative  scales  have  the  additional  advantage  that  alterna¬ 
tives  can  usually  be  scored  much  more  directly  and  meaning¬ 
fully  on  the  actual  criteria  of  interest  without  resorting 
to  surrogate  measures.  This  is  so  because  the  alternatives 
themselves  define  what  is  meant  by  such  things  as  the  best 
and  worst  reliability. 

2. 1.2. 4  Relationships  among  scaling  methods  - 
Moving  from  an  absolute  scale  to  a  ratio  scale  to  a  relative 
scale  requires  less  and  less  information.  The  absolute 
scale  requires  definitions  of  both  the  unacceptable  and 
ideal  points.  The  ratio  scale  requires  a  definition  of  only 
the  unacceptable  point;  the  best  alternative  is  used  to 
define  the  highest  point  on  the  scale.  The  relative  scale 
requires  no  definition  of  endpoints;  both  are  determined  by 
the  alternatives  under  consideration.  The  relationships 
among  the  three  scales  are  illustrated  in  Figure  2-2. 

2. 1.2. 5  Weighting  across  criteria  -  A  relation¬ 
ship  also  exists  among  the  weights  assigned  to  the  criteria 
in  the  different  methods  of  scaling.  For  example,  suppose 
that  the  decision  maker  assesses  his  tradeoffs  across  the 
absolute  scales  using  the  following  reasoning: 

o  Military  Effectiveness  is  most  important — as 
important  as  Cost  and  Reliability  combined. 
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Absolute  Ratio  Absolute  Ratio  Absolute  Ratio 

Relative  Pro-  Relative  "Military  Relative  "Reli- 

curement  Cost  Effectiveness"  ability" 


Figure  2-2 

A  COMPARISON  OF  ABSOLUTE,  RATIO,  AND 
RELATIVE  VALUE  SCALES 
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o  Cost  is  least  important — two  thirds  as  important 
as  reliability. 

These  two  judgments  specify  a  weighting  of  cost  to  military 
effectiveness  to  reliability  equal  to  20:50:30.  Remember, 
what  is  being  compared  here  is  the  relative  importance  of 
the  varying  performance  on  each  criterion  from  an  unaccept¬ 
able  level  to  an  ideal  level.  If  a  decision  maker  were  con¬ 
sistent  in  his  judgments,  he  would  specify  a  weighting  of 
12:37.5:18  for  the  ratio  scale,  because  the  range  from 
unacceptable  to  the  best  helicopter  is  60%  of  the  absolute 
range  on  cost  (.60  x  20  =  12),  75%  of  the  absolute  range  on 
military  effectiveness  (.75  x  50  *  37.5)  and  60%  of  the 
absolute  range  on  reliability  (.60  x  30  =  18).  On  the 
relative  scale,  the  decision  maker  would  specify  a  weighting 
of  10:18.75:12  because  the  range  from  the  worst  to  the  best 
helicopter  is  50%  of  the  absolute  range  on  cost  (.50  x  20  «  10), 
37.5%  of  the  absolute  range  on  military  effectiveness  (.375  x 
50  ■  18.75),  and  40%  of  the  absolute  range  on  reliability 
(.40  x  30  =  12).  Calculations  of  normalized  weights  based 
on  these  proportions  are  shown  in  Table  2-1. 

2. 1.2. 6  Evaluating  alternatives  -  Evaluations 
are  made  by  multiplying  each  alternative's  score  on  a  cri¬ 
terion  by  the  corresponding  weight  and  summing  across  cri¬ 
teria.  These  evaluations  are  shown  in  Table  2-2.  Notice 
that,  although  the  different  scales  give  different  numerical 
evaluations  for  the  alternatives,  helicopter  A  receives  the 
highest  evaluation  by  all  methods.  This  result  will  always 
occur  with  any  consistent  set  of  scores  and  weights.  Fur¬ 
thermore,  the  evaluation  of  helicopter  C  (the  second  best) 
relative  to  helicopters  A  and  B  (the  best  and  worst)  is 
exactly  the  same  regardless  of  the  evaluation  method;  that 
is,  helicopter  C's  evaluation  is  about  76%  of  the  distance 
between  B's  evaluation  and  A's  evaluation.  (Similar  results 
would  also  hold  for  any  other  helicopters  with  intermediate 


18 


Procurement  Cost  "Military  Effectiveness"  "Reliabilit 


CONSISTENT  SET  OF  NORMALIZED  WEIGHTS 


EVALUATIONS  OF  THE  HELICOPTERS 


evaluations.)  This  is  also  a  general  feature  of  the  three 
evaluation  methods. 


Some  information,  however,  is  lost  in 
moving  from  an  absolute  scale  to  a  ratio  scale  to  a  relative 
scale.  (Recall  that  this  loss  of  information  is  accompanied 
by  a  reduction  in  the  required  assessments.)  Only  the  abso¬ 
lute  evaluation  gives  information  about  how  good  the  heli¬ 
copters  are  when  compared  with  the  ideal.  For  instance, 
helicopter  A  is  52%  as  good  as  the  ideal.  Only  the  absolute 
and  ratio  evaluations  can  provide  information  on  the  percentage 
relationship  among  the  alternatives.  For  instance,  helicopter 
B  is  78%  as  good  as  helicopter  A.  However,  these  kinds  of 
comparisons,  interesting  though  they  may  be,  are  not  really 
necessary  for  a  decision.  Furthermore,  the  information  re¬ 
quired  to  make  these  comparisons  is  seldom  worth  its  cost. 

The  comparability  among  the  three 
evaluation  methods  is  further  demonstrated  in  Table  2-3. 

This  table  shows  the  simple  conversions  necessary  to  move 
from  the  absolute  evaluation  to  either  the  ratio  or  relative 
evaluation,  and  those  necessary  to  move  from  the  ratio  to 
the  relative  evaluation. 

2.2  Description  of  the  ASH  Evaluation  Model 

2.2.1  Structure  of  the  model  -  The  top  level  of  the 
model  divides  the  evaluation  into  five  main  categories  of 
values  Military  Worth,  Life  Cycle  Cost,  Attainability, 

Force  Structure  Personnel  Impact,  and  Rationalization, 
Standardization  and  Interoperability  (RSI).  This  top-level 
structure  is  illustrated  in  Figure  2-3.  The  main  categories 
of  value  are  divided  into  subcategories  to  form  a  hierarchical 
arrangement,  as  shown. 

The  first  main  category  of  value,  Military 
Worth,  is  divided  into  three  subcategories:  Operational 


CONVERSIONS  AMONG  THE  THREE  EVALUATION  METHODS 


Figure  2-3 

TOP-LEVEL  STRUCTURE  OF  THE  ASH  EVALUATION  MODEL 


Acceptability,  Technical  Systems,  and  Technical  System 
Risks.  The  first  two  subcategories  represent  complete 
evaluations  of  the  ASH  candidates  from  the  points  of  view  of 
the  users  and  the  technical  community.  The  third  category 
represents  the  risks  involved  in  achieving  the  ASH  candidates. 

This  particular  subdivision  of  Military  Worth 
was  chosen  to  accommodate  the  different  points  of  view  held 
by  users  and  technical  experts.  Each  community  is  allowed 
to  evaluate  the  candidates  in  a  way  that  seems  most  natural, 
and  neither  is  forced  to  adopt  the  other's  way  of  thinking. 
While  this  structure  may  result  in  very  different  evaluations 
of  the  candidates  by  the  different  groups  (which,  for  the 
most  part,  did  not  happen  in  this  application) ,  the  structure 
also  allows  for  a  specific  identification  of  the  reasons  for 
the  differences.  By  including  Technical  System  Risks  at 
this  level  in  the  hierarchy,  both  groups'  evaluations  can 
be  adjusted  to  reflect  the  degree  to  which  the  ability  to 
achieve  the  promised  level  of  performance  of  each  candidate 
is  uncertain. 


This  particular  division  of  the  Military  Worth 
category  is  illustrated  in  Figure  2-4.  Each  of  the  cate¬ 
gories  within  Military  Worth  is  also  subdivided.  The 
Operational  Acceptability  category  is  divided  according  to 
the  way  that  the  user  community  makes  its  evaluation.  The 
Technical  Systems  category  is  divided  in  the  manner  in  which 
the  engineers  and  designers  evaluate  the  candidates.  Technical 
Systems  Risk  is  divided  according  to  the  main  systems  of  the 
helicopters  that  are  considered  for  modification  and,  therefore, 
those  that  entail  the  greatest  degree  of  risk.  These  subdivi¬ 
sions  are  also  shown  in  Figure  2-4. 

Operational  Acceptability  is  subdivided  into 
three  main  areas:  Operational  Effectiveness,  Availability, 
and  Training.  Operational  Effectiveness  provides  a  measure 
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SUBDIVISIONS  OF  THE  MILITARY  WORTH  CATEGORY 


of  how  well  each  candidate  performs  the  scout  missions. 

Such  a  measure  is  based  both  on  the  results  of  large-scale 
simulations  (in  which  five  helicopter  designs  were  used  to 
represent  the  thirteen  ASH  candidates) ,  and  direct  judgmental 
evaluations  of  the  performance  of  the  candidates  in  the 
major  scout  missions  (considering  those  functions  that  are 
important  to  each  mission) .  Subdivisions  of  Operational 
Effectiveness  are  detailed  in  Figure  2-5.  While  Operational 
Effectiveness  measures  the  performance  of  the  candidate  when 
it  is  in  action,  Availability  measures  the  aspects  that  are 
important  to  keeping  each  helicopter  ready  for  action. 
Subdivisions  of  Availability  are  detailed  in  Figure  2-6. 

The  area  of  Training  is  subdivided  by  the  type  of  personnel 
involved — the  helicopter's  crew  and  maintenance  personnel. 

Value  from  a  Technical  System  standpoint  is 
subdivided  according  to  the  main  types  of  systems  (mission 
equipment  and  airframe)  and  the  degree  to  which  ASH  candi¬ 
dates  integrate  the  systems  effectively.  Detailed  sub¬ 
divisions  of  the  Technical  Systems  are  are  shown  in  Figure  2-7. 

The  second  main  category  of  value  is  Life  Cycle 
Cost  (LCC) .  Subdivisions  of  LCC  correspond  to  the  timing  of 
the  cost  as  well  as  the  specific  type  of  funding  required. 

Thus,  LCC  is  divided  into  Acquisition  Cost  and  Ownership 
Cost.  Acquisition  Cost,  in  turn,  is  divided  into  RDT&E  Cost 
and  Procurement  (APA)  Cost.  Ownership  Cost  is  divided  into 
other  anticipated  investments  and  Operations  &  Support. 
Subdivisions  of  LCC  are  shown  in  Figure  2-8. 

The  third  main  category  of  value  is  Attain¬ 
ability.  This  category  measures  the  degree  to  which  each 
ASH  candidate  is  affordable  and  is  able  to  meet  schedule 
constraints.  Affordability  mainly  concerns  the  relationship 
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SUBDIVISIONS  OF  LIFE  CYCLE  COSTS 


between  each  candidate's  funding  requirement  and  the  avail¬ 
ability  of  funds  as  reflected  in  the  Army's  latest  Program 
Objectives  Memorandum  (POM) .  The  Schedule  subcategory  con¬ 
cerns  each  candidate's  ability  to  meet  the  Initial  Opera¬ 
tional  Capability  (IOC)  mandated  by  Congress  as  well  as  its 
entire  production  schedule.  Both  the  cost  estimates  and 
schedule  estimates  are  adjusted  by  their  riskiness,  as 
reflected  in  the  third  subcategory.  Subdivisions  of  Attain¬ 
ability  are  detailed  in  Figure  2-9. 

The  fourth  main  category  of  value.  Force  Struc¬ 
ture  Personnel  Impact,  reflects  the  increased  personnel 
requirements  of  the  ASH  candidates. 

Rationalization,  Standardization,  and  Interoper¬ 
ability,  the  final  main  category  of  value,  is  concerned 
with  the  possibilities  for  joint  development  and  production 
of  ASH  candidates  with  allies  (especially  NATO  allies)  and 
the  ability  of  the  ASH  candidates  to  perform  missions  in 
conjunction  with  allied  military  units.  Details  of  the 
suodivisions  of  this  category  are  shown  in  Figure  2-10. 

2.2.2  Assessed  inputs  to  the  ASH  Evaluation  Model  - 
The  ASH  alternatives  are  evaluated  by  using  a  relative 
scoring  model,  as  explained  in  Sections  2.1.1  and  2.1.2. 

Since  detailed  displays  of  all  inputs  to  the  model  are  given 
in  the  computer  printouts  in  Appendices  A  and  B,  selected 
inputs  are  highlighted  in  the  following  paragraphs. 

Most  of  the  subcategories  of  value  (the  bottom- 
level  branches  in  the  structure  described  in  Section  2.2.1) 
are  areas  of  interest  that  are  not  readily  quantifiable  on  an 
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underlying  scale.  For  example,  consider  the  following  path 
in  the  structure  (from  Figures  2-4  and  2-7): 

Military  Worth 

Technical  Systems 

Mission  Equipment  Package 

V  * 

Target  Acquisition/Designation  System. 

This  path  is  entry  level  1.1. 2. 1.3  in  the  model.  The  tech¬ 
nical  appraisal  of  the  value  of  various  target  acquisition/ 
designation  systems  (TA/DS)  is  based  directly  on  the  quality 
of  the  equipment  involved.  Quality  is  determined  by  such 
factors  as: 

o  the  types  of  sensors  used; 

o  the  survivability  of  the  system  based  on  its 

mounting  location  (mast-mounts  are  more  survivable 
than  nose-mounts) ; 

o  the  range  of  the  sensors;  and 

o  the  capability  of  the  sensors  under  condition  of 
limited  visibility  (especially  night  operations. 

Considering  these  factors,  seven  of  the  thirteen 
ASH  candidates  are  judged  to  have  the  best  TA/DS:  BTA,  350, 
129,  BT2 ,  BTT,  OHM,  and  B4K.  These  seven  systems  are  all 
assigned  scores  of  100.  The  OH-58C  is  judged  to  have  the 
worst  TA/DS  and  is  assigned  a  score  of  zero.  Other  candidates 
have  intermediate  levels  of  performance  on  TA/DS  and  are  scored 
accordingly.  For  instance,  the  improvement  in  TA/DS  on  the 
OH-58D  over  the  OH-58C  is  assessed  to  be  about  40%  of  the  BTA' s 
improvement,  so  the  OH-58D  receives  a  score  of  40.  Similar 
reasoning  leads  to  the  scores  for  the  other  ASH  candidates 
given  in  Table  2-4.  (Detailed  rationale  for  each  score  is 
given  in  Appendix  B.) 
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Of  course,  some  of  the  entry-level  branches  in 
the  structure  lend  themselves  to  quantification  on  the  basis 
of  some  natural  unit,  such  as  dollars  for  cost.  In  these 
cases,  estimates  can  be  made  on  the  natural  scales  and,  then 
transformed  to  relative  scales  for  consistency.  For  example, 
estimates  of  the  APA  (procurement)  costs  for  the  candidates 
range  from  a  low  of  $138,000  per  unit  for  the  OH-58C  to  a 
high  of  $3,857  million  for  the  OH-64.  (Cost  estimates  are 
"most  likely"  costs  based  on  "prime  quantity"  orders,  gener¬ 
ally  1472,  stated  in  fiscal  year  1980  dollars.)  The  relative 
scores  for  these  candidates  are  then  100  for  the  OH-58C,  to 
signify  that  it  has  the  most-preferred  cost,  and  0  for  the 
OH-64.  Other  ASH  candidates  are  scored  at  intermediate 
levels  in  proportion  to  their  costs. 

The  next  task  in  working  up  through  the  hier¬ 
archy  is  to  trade  off  the  scores  across  the  criteria.  In 
this  operation,  the  importance  of  the  range  of  impact  of  the 
candidates  on  one  criterion  is  traded  off  against  the  impor¬ 
tance  of  the  range  of  impacts  on  other  criteria.  These 
trade-offs  are  expressed  as  weights  assessed  for  each  criterion. 

Consider  the  category  of  Mission  Equipment 
Packages,  which  is  the  aggregation  of  navigation,  communica¬ 
tions,  and  TA/DS  equipment,  together  with  equipment  growth 
possibilities.  The  following  line  of  reasoning  leads  to  the 
weights  assessed  for  each  criterion  as  displayed  in  Table  2-5: 

o  The  range  of  impacts  of  the  ASH  candidates 
on  TA/DS  is  most  important. 

o  The  range  of  impacts  on  TA/DS  is  as  important 
as  the  impacts  on  Navigation  and  Communications 
combined. 
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The  improvement  of  BTA's  communications  equipment 
over  58C's  communications  equipment  is  equally  as 
important  as  the  improvement  of  BTA's  navigation 
equipment  over  500's  navigation  equipment. 

o  The  range  of  impacts  on  Growth  is  one-half  as 
important  as  the  range  of  impacts  on  Communi¬ 
cations. 

These  relationships  establish  the  set  of  weights  for  the 
criteria  shown  in  Table  2-5  (the  weights  are  normalized  to 
add  to  100%  for  consistency.) 

The  total  evaluation  of  each  candidate  in  the 
category  of  Mission  Equipment  is  obtained  by  taking  a  weighted- 
average  of  the  candidate's  scores  in  the  subcategories.  For 
example,  BTA's  total  evaluation  is  calculated  as  follows: 

(.25) (80)  +  (.20)  (100)  +  (.45)  (100)  +  (.10)  (80)  *  93. 

These  weighted-average  evaluations  are  combined,  in  a  similar 
manner,  with  weighted-average  evaluations  from  other  categories 
to  arrive  at  evaluations  at  higher  levels  in  the  structure. 

Table  2-6  shows  the  evaluations  of  the  candidates 
on  each  major  category  of  value.  At  least  two  types  of  analy¬ 
sis  can  be  done  with  these  evaluations:  trade-off  weights 
could  be  assessed  across  the  major  categories  to  arrive  at  a 
single  "best"  candidate,  or  the  evaluations  on  certain  cate¬ 
gories  (such  as  Military  Worth)  could  be  plotted  against 
cost  to  identify  the  efficient  set  of  candidates— those  that 
provide  the  most  benefit  at  different  levels  of  cost. 
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Table  2-7  displays  the  overall  evaluations  of 
the  ASH  candidates  based  on  the  following  assessment  of  weights: 

Military  Worth  50% 

Life  Cycle  Cost  30% 

Attainability  15% 

Force  Structure  Personnel  Impact  3% 

Rationalization,  Standardization,  &  Interoperability  2%. 

Overall  evaluations  now  range  from  a  high  of  61  points  for 
the  BTA  to  a  low  of  43  for  the  OH-64.  So,  with  these  trade¬ 
offs  across  criteria,  BTA  is  the  preferred  candidate. 

Rather  than  make  all  of  these  trade-offs  across 
the  five  top-level  categories  of  value,  one  might  wish  to 
examine  the  efficiency  of  the  candidates  as  a  function  of 
one  of  the  categories.  Of  particular  interest  is  the  plot 
of  Military  Worth,  Attainability,  Force  Structure  Personnel 
Impact,  and  RSI  (weighted  in  the  same  proportion  as  in  Table 
2-7)  versus  Utility  for  Life  Cycle  Cost.  Such  a  plot  allows 
identification  of  the  most  "beneficial"  candidates  at  various 
levels  of  "cost,"  where  "cost"  is  determined  by  weighting 
dollar  costs  of  different  categories  (such  as  RDT&E  cost) 
according  to  the  importance  of  saving  money  in  each  category 
(as  reflected  in  the  weights  assigned  to  each  cost  category, 
see  Appendix  A).  Plotting  these  values  leads  to  Figure  2-11. 

Figure  2-11  is  arranged  so  that  the  efficient 
ASH  candidates  are  those  appearing  on  the  upper  left  (or 
northwest)  edge  of  the  plot.  These  candidates  are  the  BTA, 
OH-58E,  OH-58D,  and  OH-58C.  (Recall  that  utilities  for  cost 
are  assigned  so  that  low  costs  receive  high  scores  on  0-to- 
10C  scales.  Thus,  the  transformation  of  100-Cost  Utility 
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CANDIDATES  VERSUS  COST 


coordinates  low  scores  with  low  costs) .  The  following 
reasoning  explains  why  these  efficient  candidates  dominate 
the  others: 

o  BTA  is  both  better  and  cheaper  than  350 , 

129,  BT2,  BTT,  OHM,  and  064. 

o  BTA  is  cheaper  than  B4K  and  both  provide 
equal  benefit. 

o  58E  is  better  and  cheaper  than  OHT. 

o  58D  is  better  and  cheaper  than  500. 

A  similar  plot  results  when  dollar  cost  (rather 
than  cost  utility)  is  plotted  on  the  horizontal  axis  or  when 
Military  Worth  alone  is  plotted  on  the  vertical  axis. 

Figure  2-12  shows  the  plot  that  results  when  both  of  these 
changes  are  made.  The  only  differences  in  the  efficient 
candidates  are  that  the  500  and  the  B4K  become  efficient 
when  benefit  is  determined  by  just  Military  Worth,  and 
equal-dollar  costs  of  all  categories  are  weighted  equally. 

An  interesting  and  important  feature  of  both 
Figures  2-11  and  2-12  is  that  the  efficient  candidates 
almost  lie  on  a  straight  line.  This  feature  is  important 
because  it  means  that  all  of  the  efficient  candidates  are 
about  equally  efficient;  that  is,  "you  get  what  you  pay 
for."  In  such  a  case,  the  required  level  of  benefit  or  the 
amount  of  money  available  might  be  the  best  determinant  of 
the  "optimal"  candidate. 

2.2.3  Sensitivity  analyses  -  Numerous  sensitivity 
analyses  were  conducted  during  the  course  of  this  study. 
These  analyses  were  aimed  at  determining  the  assessments 
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of  both  scores  and  weights  in  the  model  that  most  critically 
drove  the  results;  this  was  done  as  a  way  to  identify  im¬ 
provements  in  the  model.  These  analyses  involved  changes  in 
both  the  scores  of  the  alternatives  on  criteria  and  the 
weights  across  the  criteria.  Since  the  results  of  the 
analyses  led,  for  the  most  part,  to  refinements  of  the 
model's  inputs  and  structure  that  made  older  versions  of  the 
model  obsolete,  few  records  were  kept  of  the  analyses;  and 
the  discarded  models  are  of  little  interest.  For  these 
reasons,  we  will  not  attempt  to  reconstruct  those  analyses 
but  instead  will  present  a  few  illustrative  examples. 

One  type  of  sensitivity  analysis  examines  the 
effects  on  the  overall  evaluations  of  changes  in  the  weight 
assigned  to  a  particular  criterion.  For  example.  Table  2-8 
shows  the  results  of  varying  the  weight  assigned  to  Military 
Worth.  At  the  current  weight,  50%,  the  alternatives  receive 
the  same  evaluations  shown  in  Table  2-7.  In  particular,  BTA 
receives  the  highest  overall  evaluation,  61  points.  These 
evaluations  are  displayed  on  the  row  labeled  50.0  on  Table 
2-5,  and  BTA's  highest  evaluation  is  marked  with  an  asterisk. 
Table  2-8  shows  that  BTA  receives  the  highest  evaluation  as 
long  as  the  weight  assigned  to  Military  Worth  remains  between 
50%  and  70%.  For  weights  above  70%,  B4K  is  best  and  for 
weights  at  or  below  40%,  58C  is  best.  Since  B4K  scores 
highest  in  Military  Worth,  it  is  not  surprising  that  B4K  is 
most  preferred  when  the  weight  assigned  to  Military  Worth  is 
very  high.  Similarly,  58C  is  best  on  the  composite  of 
everything  except  Military  Worth  (most  importantly  Life 
Cycle  Costs) ,  so  it  is  no  surprise  that  it  is  preferred  for 
low  weights  on  Military  Worth. 

Analyses  of  this  type  can  also  be  performed  at 
other  levels  in  the  structure.  For  example,  Table  2-9  shows 
the  sensitivity  of  the  evaluations  to  changes  in  the  weight 
assessed  for  Mission  Equipment  Packages.  In  this  table,  the 
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cumulative  weight  assigned  to  Mission  Equipment,  which  is 
calculated  by  multiplying  all  of  the  weights  down  the  path 
in  the  structure,  is  varied  from  0  to  50%  of  the  total  in 
the  model  (its  current  weight  is  13.5%.)  Within  this  range 
of  variation,  the  preferred  candidate  shifts  from  58C,  for 
very  low  weights,  to  BTA  for  higher  weights. 

Sensitivity  of  the  evaluations  to  changes  in 
other  inputs,  such  as  combinations  of  weights  and  scores, 
can  be  readily  determined  by  entering  new  values  and  having 
the  computerized  model  calculate  the  new  evaluations.  This 
type  of  analysis  was  done  for  a  variety  of  changes  during 
the  course  of  the  project. 

Another  type  of  analysis  also  proves  useful:  a 
discrimination  analysis  that  identifies  and  sorts  the  dif¬ 
ferences  between  candidates  that  contribute  to  the  dif¬ 
ferences  in  their  evaluations.  Table  2-10  displays  such  a 
comparison  of  the  OH-58D  and  the  Hughes  500D  (comparisons  of 
other  pairs  of  candidates  are  given  in  Appendix  C) .  This 
analysis  shows  that  the  most  important  single  discriminator 
favoring  the  OH-58D  is  APA  (procurement)  cost,  which  contrib 
utes  4.2  points  to  the  overall  difference  between  the  two 
candidates.  Reading  down  the  list,  one  sees  that  many  of 
the  subcategories  of  Attainability  also  favor  the  58D,  as  do 
the  cost  subcategories  of  Accident  Life  Cycle  Cost  and  Other 
Investment  Cost.  In  the  middle  range  of  the  chart  are  dis¬ 
played  all  of  the  subcategories  in  which  the  performance  of 
the  two  candidates  is  the  same.  These  include  most  of  sub¬ 
categories  of  Military  Worth.  The  subcategories  in  which 
the  500  is  preferred  to  the  58D  are  given  at  the  bottom  of 
the  table.  Most  important  of  these  is  Other  Operations  and 
Support  Cost,  which  favors  the  500  by  1.7  overall  points. 
Other  major  differences  favoring  the  500  include  subcate- 
gories  of  the  technical  performance  of  the  airframe  and  most 
subcategories  of  RSI. 
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Comparisons  such  as  the  one  in  Table  2-10  can  be 
used  in  several  ways.  Most  obviously,  they  can  be  used  to 
explain  the  reasons  for  the  differences  in  overall  evaluations. 
More  importantly,  though,  these  types  of  analyses  can  be 
used  to  suggest  the  types  of  changes  in  criterion  weights 
that  are  required  to  change  the  relative  evaluations  of  the 
candidates.  For  instance.  Table  2-10  suggests  that  increases 
in  the  importance  of  RSI  would  tend  to  improve  the  500's 
evaluation  relative  to  that  of  the  58D,  as  would  increases 
in  the  weight  assigned  to  Other  Operations  and  Support  Cost 
or  Airframe  Performance.  On  the  other  hand,  increases  in 
the  weights  assigned  to  any  other  cost  category  would  in¬ 
crease  the  58D's  preference  over  the  500,  as  would  an  in¬ 
crease  in  the  weight  assigned  to  Attainability.  Changes  in 
weights  assigned  to  Operational  Effectiveness  or  to  most 
Technical  Systems  (other  than  airframe  performance)  would 
not  change  the  relative  evaluations  of  those  two  candidates. 

2.2.4  Mixes  of  ASH  Candidates  -  In  addition  to  the 
thirteen  ASH  candidates  discussed  above,  the  Special  Study 
Group  is  also  interested  in  evaluating  mixes  of  ASH  candidates. 
These  mixes  use  one  helicopter  to  perform  a  high-capability 
mission  role  and  a  different  one  to  perform  a  low-capability 
mission  role  and  training  and  reserve  roles.  Two  distinct 
models  were  used  to  evaluate  different  types  of  mixes.  The 
first  modex  evaluates  four  mixes  of  ASH  candidates,  assuming 
that  the  total  number  of  helicopters  to  be  purchased  remains 
at  1472.  The  second  model  evaluates  seven  new  mixes  that 
vary  the  total  quantity  of  helicopters  as  well  as  the  mixtures. 

2. 2. 4.1  Mixes  involving  1472  helicopters  -  The 
first  set  of  mixes  evaluated  assumed  that  343  helicopters 
would  be  procured  to  serve  for  the  high-capability  role,  740 
helicopters  would  be  procured  for  the  low-capability  role, 
and  389  helicopters  would  be  procured  for  the  training  and 
reserve  roles.  This  gives  a  total  procurement  figure  of 
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1472,  which  is  the  sane  size  assuned  for  the  honogeneous 
purchases  of  ASH  candidates  evaluated  above. 


following: 


The  nixes  to  be  evaluated  included  the 


o 

343 

BTA  plus  1129  OH-58C 

(B+C) 

o 

343 

BTA  plus  1129  OH-58D 

(B+D) 

o 

343 

OH-64  plus  1129  OH-58C 

(64C) 

o 

343 

OH-64  plus  1129  OH-58D 

(64D)  . 

The  nodel  that  evaluates  these  nixes 
uses  the  results  of  the  nodel  of  honogeneous  packages  as  a 
basis  for  sinplification.  Most  inportantly,  the  Military 
Worth  of  each  nix  is  determined  by  aggregating  the  Military 
Worth  evaluations  of  its  conponents  as  detemined  by  the 
original  nodel.  Table  2-11  shows  the  sinplified  nethod  that 
is  used  to  deternine  the  Military  Worth  of  the  nixes.  First, 
each  nix  is  scored  on  its  ability  to  perfom  the  high  role. 
This  is  sinply  the  score  of  the  candidate  that  fills  the 
high  role  as  deternined  by  the  nodel  displayed  on  Table  2-7. 
Thus,  nixes  that  have  the  BTA  in  the  high  role  receive 
Military  WTorth  scores  of  76,  and  those  that  have  the  OH-64 
in  the  high  role  receive  scores  of  60.  A  similar  step 
determines  the  score  of  each  mix  in  the  low  role;  the  score 
is  that  of  the  candidate  filling  the  low  role  as  determined 
by  the  nodel  in  Table  2-7.  Thus,  nixes  that  use  the  OH-58C 
receive  scores  of  24,  and  those  that  use  the  OH-58D  receive 
scores  of  36.  (The  analysts  questioned  whether  the  scores 
of  the  helicopters  serving  in  the  low  role  should  be  raised 
to  reflect  a  lesser  required  capability.  The  users  responded, 
however,  that  the  scores  of  these  particular  candidates 
should  not  be  raised.)  Next,  weights  were  assessed  across 
the  high  and  low  roles.  These  weights,  which  are  displayed 
in  Table  2-11,  are  based  on  the  assessnent  that  the  range  of 
inpacts  of  the  candidates  on  fulfilling  the  high  role  is 
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1.1 

-  ASH 

-  MIL  WORTH 

FACTOR 

WT 

B+C 

B+D 

64C 

64D 

1) 

MW  HIGH 

*(  55) 

76 

76 

60 

60 

2) 

MW  LOW 

#(  45) 

24 

36 

24 

36 

TOTAL 

t 

53 

58 

44 

49 

Weights  based  on: 

(1)  High  role  is  much  more  important 

(2)  More  helicopters  in  low  role  (343  high 

versus  740  low) 


Table  2-11 

MILITARY  WORTH  ASSESSMENTS  FOR  MIXES 
OF  1472  HELICOPTERS 
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about  twice  as  important  as  the  range  of  impacts  on  the  low 
role.  This  assessment  was  then  adjusted  to  account  for  the 
greater  number  of  helicopters  that  will  fulfill  the  low 
role.  (The  adjustment  was  made  judgmentally  in  this  evalua¬ 
tion  but  was  formalized  in  the  following  one.) 

Scores  of  the  mixes  on  Force  Structure 
Personnel  Impact  and  RSI  were  also  estimated  based  on  the 
previous  evaluation  model.  However,  in  the  categories  of 
Life  Cycle  Costs  and  Attainability,  scores  were  assessed  at 
all  of  the  end-branches  of  the  original  model.  The  struc¬ 
tures  for  these  scores  are  shown  in  Figures  2-8  and  2-9. 

The  evaluations  of  the  mixes  can  then 
be  displayed  on  the  same  plot  as  Figure  2-11,  which  is  done 
in  Figure  2-13.  Examination  of  this  plot  reveals  that  only 
one  of  the  mixes  is  efficient:  the  BTA  coupled  with  the 
58C.  This  efficient  mix  offers  about  one-half  of  BTA's 
improvement  over  58C  at  about  one-half  of  the  corresponding 
increase  in  "cost." 


2. 2. 4. 2  Mixes  with  different  numbers 
of  helicopters  -  Later  discussions  resulted  in  another  set 
of  mixes  to  be  evaluated.  Mixes  in  this  set  varied  in  the 
number  and  types  of  helicopters  serving  the  high  role  and 
low  role  as  follows: 


Mix  High  Role 


Low  Role 


A 

1 

2 

3 

4 

5 

6 
7 


750  BTA 
615  BTA 
490  BTA 
363  BTA 
490  BTA 
363  BTA 
270  OH-64 
264  OH-64-M 


0 

468  OH-58D+ 
593  OH-58D+ 
720  OH-58D+ 
260  OH-58D+ 
387  OH-58D+ 
387  OH-58D+ 
387  OH-58D+ 
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Mix  "A"  is  really  not  a  combination  of  two  ASH  alternatives, 
but  simply  a  reduced  order  of  the  BTA.  Mixes  2  and  3  total 
1083  helicopters  and  use  various  numbers  of  BTAs  in  the  high 
role  and  OH-58D+S  in  the  low  role.  (The  OH-58D+  is  an  en¬ 
hanced  version  of  the  OH-58D) .  Mixes  4  and  5  use  fewer 
(750)  of  the  same  helicopters.  Mixes  6  and  7  substitute 
versions  of  the  OH-64  for  the  BTA  and  are  targeted  to  cost 
the  same  (in  APA  dollars)  as  mix  5.  Mix  6  uses  the  same 
OH-64  evaluated  in  Section  2.2.2,  which  has  a  nose-mounted 
sight,  and  mix  7  uses  a  modification  of  the  OH-64  that  has  a 
mast-mounted  sight. 


Since  quick  evaluations  were  required 
for  these  mixes,  a  simplified  model  was  used.  The  model 
considers  only  Military  Worth  and  Cost.  (Since  Figures  2-11 
and  2-12  are  so  similar  to  each  other,  it  seemed  certain 
that  an  analysis  which  used  only  Military  Worth  would  pro¬ 
duce  a  result  very  similar  to  that  of  an  analysis  that 
involved  a  more  comprehensive  measure  of  benefit) .  The 
scores  for  the  mixes  on  Military  Worth  are  based  on  those  in 
the  original  model,  but  are  adjusted  for  the  number  of  units. 

The  scores  of  the  mixes  are  shown  in 
Table  2-12.  The  scores  for  the  mixes  containing  the  BTA  in 
the  high  role  begin  with  its  assessed  Military  Worth  of  76. 
This  figure  is  then  adjusted  according  to  the  number  of 
units,  using  750  as  an  estimate  of  the  number  required  to 
serve  the  role  completely.  For  instance,  the  mix  with  615 
BTA's  receives  a  score  of: 

76  *  7?l  -  «• 

The  mix  that  contains  the  ordinary  OH-64  is  scored  by  taking 
a  proportion  of  the  Military  Worth  score  for  the  OH-64 
determined  in  the  original  analysis,  60.  The  last  mix  is 
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scored  beginning  with  an  assessment  of  the  score  for  the 
OH-64-M.  This  score  is  estimated  at  71,  which  is  slightly 
above  the  score  for  the  OHM;  the  OH-64-M  is  more  survivable. 
These  scores  are  then  adjusted  for  the  number  of  units 
involved. 

Similar  adjustments  are  made  for  scores 
in  the  low  role.  First,  a  score  is  assessed  for  the  OH-58D+. 
This  candidate  was  judged  to  be  about  as  good  as  the  58E  for 
the  low  role.  So,  the  basic  Military  Worth  score  for  the 
OH-58D+  was  assessed  at  58  (from  Table  2-6) .  The  score  for 
each  mix  is  calculated  by  adjusting  this  score  by  the  number 
of  units,  assuming  that  720  are  required  to  fully  perform 
the  low  role.  For  example,  the  mix  that  contains  468  OH- 
58D+s  receives  a  score  of : 


58  x 


The  evaluations  of  the  mixes  on  the 
basis  of  Military  Worth  are  calculated,  as  before,  by  taking 
a  weighted  average  of  the  scores.  In  this  case,  since  the 
scores  reflect  the  number  of  units  to  be  procured,  the 
proper  weights  are  those  that  reflect  the  importance  of  the 
range  of  impacts  of  the  candidates  on  the  criteria.  Specifi¬ 
cally,  the  high  role  should  receive  a  weight  twice  that  of 
the  low  role. 

Cost  comparisons  for  these  mixes  are 
best  performed  on  a  total  basis--rather  than  per  unit— 
because  different  mixes  involve  different  quantities.  Es¬ 
timates  were  made  of  the  amount  of  each  category  of  cost  at 
different  levels  of  procurement.  Both  the  total  cost  and 
the  APA  costs  of  the  mixes  are  given  in  Table  2-12.  These 
evaluations  can  be  plotted  as  shown  in  Figures  2-14  and  2-15. 
Figure  2-14  shows  that  the  BTA  is  more  efficient  than  any 
of  the  mixes  on  the  basis  of  total  cost.  However,  if  all 
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costs  other  than  APA  (procurement)  cost  are  ignored,  then 
the  mix  of  363  BTA's  and  387  OH-58D+'s  is  also  efficient,  as 
shown  in  Figure  2-15. 

2.3  An  Alternative  Interpretation  of  the  ASH  Evaluation 
Model 


Sections  2.1  and  2.2  explain  the  methodology  used  in 
constructing  an  evaluation  model  and  describe  details  of  the 
model  built  to  evaluate  ASH  candidates.  This  section  de¬ 
scribes,  in  a  series  of  charts,  an  alternative  way  of  inter¬ 
preting  the  model.  This  interpretation  begins  by  taking  the 
results  of  the  COEA  simulations  (CARMONETTE  and  AVWAR)  as  an 
initial  evaluation  of  the  ASH  candidates  and  then  adjusts 
this  evaluation  to  reflect  items  that  were  not  included  in 
the  simulations. 

The  first  adjustment  considers  individual  differences 
among  the  13  candidates  that  were  not  modelled  in  the  simu- 

i 

lation  (such  as  the  field  artillery  mission) .  The  second 
adjustment  reflects  the  equipment  evaluations  made  by  the 
technical  community.  The  third  adjustment  factors  in  con¬ 
siderations  of  technical  risk  and  training  requirements. 

The  fourth  adjustment  accounts  for  Attainability,  Force 
Structure  Personnel  Impact,  and  RSI.  The  result  of  these 
adjustments  is  an  evaluation  of  each  candidate  on  all  attri¬ 
butes  of  value  except  cost.  This  evaluation  is  the  same  one 
that  is  plotted  against  the  utility  for  cost  in  Figure  2-13. 
These  steps  are  displayed  on  the  following  charts. 
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3.0  THE  DESIGN  MODEL 


3 . 1  Introduction 


In  addition  to  the  evaluation  model  described  in 
Section  2,  some  effort  was  made  to  identify  efficient  ASH 
configurations  using  a  Design  Model.  Such  a  model  identi¬ 
fies  important  factors  and  assesses  the  cost/benefit  trade¬ 
offs  of  improvements  on  each  factor.  For  a  variety  of 
reasons,  which  are  mentioned  in  Section  3.4,  this  model 
proved  to  be  inappropriate  for  the  task  at  hand  and  was 
abandoned  in  favor  of  the  exclusive  use  of  the  evaluation 
model.  Nevertheless,  the  Design  Model  has  been  included  in 
this  report  for  completeness  and  as  an  illustration  of  a 
type  of  analysis  that  could,  under  different  circumstances, 
be  useful. 

In  general,  a  Design  Model  serves  the  following  purposes 

(1)  it  enumerates  a  set  of  design  options; 

(2)  it  identifies  the  efficient  (most  cost-beneficial) 
designs;  and 

(3)  it  evaluates  the  efficiency  of  any  proposed  design 
and  suggests  improvements. 

While  a  Design  Model  is  ideal  for  determining  whether  a 
proposed  design  can  be  improved,  it  is  not  especially  useful 
for  evaluating  a  specified  set  of  designs. 
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3 . 2  The  Design  Methodolo 


The  purpose  of  DDI's  Design  methodology  is  to  assist  in 
the  identification  of  efficient  designs.  There  are  three 
steps  to  this  analysis: 

Step  1  The  variables  that  can  differ  between  de¬ 
signs,  and  the  specific  levels  over  which 
they  vary,  are  identified. 

Step  2  Three  quantities  are  determined: 

the  cost  of  an  improvement  on  each 
variable, 

the  benefit  associated  with  each  im¬ 
provement  within  a  variable,  and 

-  the  relative  importance  across  variables. 

Step  3  The  cost/benefit  trade-offs  are  examined 

with  an  eye  towards  discovering  cheaper  de¬ 
signs  that  yield  the  same  benefit  as  any  pro¬ 
posed  design  and  better  designs  for  the  same 
cost  as  the  proposed  design. 

Steps  1  and  2  require  the  judgment  of  experts  and  are  used 
as  input  to  DDI's  DESIGN  software,  which  carries  out  Step  3. 

3.2.1  The  model  structure  -  The  basic  component  of  a 
Design  Model  is  a  variable.  Variables  represent  choices 
that  the  designer  can  make.  For  instance,  ASH  could  be  de¬ 
signed  to  be  capable  of  differing  speeds  or  it  could  be 
designed  with  different  navigation  equipment.  Each  of  these 
factors,  e.g.,  speed  and  navigation  equipment,  is  a  variable 
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that  can  distinguish  one  design  from  another.  The  first 
part  of  a  design  analysis  is  to  identify  a  set  of  variables 
that  influence  the  cost  or  effectiveness  of  a  design. 

Once  the  variables  have  been  identified,  it  is 
necessary  to  specify  the  levels  over  which  they  vary.  This 
is  done  by  identifying  a  minimal  level  that  must  be  achieved, 
a  maximal  level  that  is  as  high  as  one  could  reasonably 
expect,  and  intermediate  levels  that  offer  more  moderate 
design  options.  For  instance,  with  the  design  variable 
Speed  it  was  known  that  even  a  minimal  ASH  candidate  (OH- 
58C)  is  capable  of  115  knots,  while  speeds  in  excess  of  195 
knots  do  not  add  any  benefit.  Levels  of  145  and  160  knots 
were  included  to  reflect  the  possibility  of  intermediate 
designs.  Thus,  Speed  could  vary  over  four  levels:  115, 

145,  160,  and  195  knots. 

Sometimes,  when  it  is  difficult  for  the  experts 
to  generate  design  variables,  they  are  asked  to  system¬ 
atically  compare  hypothetical  minimal  and  maximal  designs. 

The  minimal  design  has  those  features  that  are  absolutely 
necessary;  the  maximal  design  is  "gold-plated,"  containing 
all  of  the  characteristics  that  one  would  like  it  to  have. 
This  procedure  helps  to  generate  variables. 

One  major  restriction  on  the  Design  methodology 
is  that  the  variables  must  be  independent  of  each  other.  In 
other  words,  it  must  be  reasonable  to  speak  of  each  level  on 
the  other  variables.  When  this  restriction  is  violated,  the 
design  software  might  suggest  optimal  designs  that  are 
impossible. 


Once  the  set  of  design  variables  and  the  levels 
of  each  are  known,  designs  can  be  generated  by  choosing  one 
level  on  each  variable.  If  the  least  desirable  level  on 


each  variable  is  chosen,  then  the  minimal  design  is  described. 
If  the  most  desirable  levels  are  chosen,  then  the  maximal 
design  is  described.  Intermediate  designs  can  be  constructed 
by  specifying  high  levels  on  some  variables  and  low  levels 
on  others. 


3.2.2  Assessing  benefits  and  costs  -  Benefits  are 
assessed  for  each  variable  independently.  The  lowest  level 
is  assigned  a  value  of  0  and  the  highest  a  value  of  100. 
Intermediate  levels  are  assigned  values  reflecting  the  per¬ 
cent  of  the  maximum  attainable  improvement  that  they  pro¬ 
vide.  The  relative  importance  of  each  variable  in  terms  of 
its  contribution  to  benefit  is  reflected  by  a  weight  associ¬ 
ated  with  each  variable.  This  weight  reflects  the  benefit 
of  going  from  the  miminal  to  the  maximal  level  on  the  variable. 
l  The  most  important  variable  is  usually  assigned  »  weight  of 
100.  Then,  the  other  variables  are  assigned  weights  that 
reflect  their  importance  as  measured  against  the  maximum 
improvement  on  the  most  important  variable.  For  instance, 
if  navigation  equipment  is  assigned  a  weight  of  70  and  TADS 
equipment  a  weight  of  100,  this  would  imply  that  an  improve¬ 
ment  from  the  lowest  to  the  highest  level  on  navigation 
equipment  is  worth  70%  of  the  benefit  derived  from  an  improve¬ 
ment  from  the  lowest  to  the  highest  level  on  TADS  equipment. 
Later,  these  weights  are  normalized  to  add  to  100.  (This 
assessment  of  benefits  is  consistent  with  the  "relative" 
method  discussed  in  Section  2.1.) 

Costs  are  also  assessed  for  each  level  of  each 
variable;  in  cases  where  actual  costs  of  various  levels  are 
known,  these  figures  are  entered  into  the  model.  In  other 
cases,  the  model  can  use  incremental  costs  of  moving  from 
one  level  to  another.  As  with  the  benefits,  the  model 
accommodates  variables  with  independent  costs.  If  cost  de¬ 
pendencies  exist  among  the  variables,  then  the  model  is 
restructured  to  achieve  independence. 
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3.2.3  Exercising  the  Design  model  -  The  procedure  for 
calculating  the  total  costs  and  total  benefits  of  any  par¬ 
ticular  design  is  to  add  them  up,  respectively.  Thus,  the 
total  cost  of  a  particular  design  is  the  sum  of  the  costs 
associated  with  each  level  on  each  variable  comprising  it. 
The  total  benefit  of  a  particular  design  is  the  sum  of  its 
weighted  benefit  scores. 

If  the  set  of  all  possible  designs  is  plotted  on 
a  cost/benefit  graph,  the  designs  will  tend  to  fall  within 
the  lens-shaped  region  depicted  in  Figure  3-1.  (Note  that 
the  costs  are  rescaled  to  add  to  100  for  this  display.)  The 
solid  upper  line  is  called  "the  frontier."  It  indicates  the 
designs  which  are  efficient,  i.e.,  those  that  produce  the 
most  benefit  for  any  given  cost.  Often,  a  proposed  design 
will  fall  somewhere  below  the  frontier,  such  as  at  the  point 
marked  P.  When  this  happens,  it  is  possible  to  find  a  de¬ 
sign  that  costs  the  same  amount,  but  yields  more  benefit 
(marked  B  for  better) ,  as  well  as  a  design  that  yields  as 
much  benefit  and  costs  less  (marked  C  for  cheaper) .  Careful 
consideration  of  these  preferable  designs  represents  the 
heart  of  the  design  evaluation. 

The  primary  caution  about  the  Design  methodology 
is  that  variables  must  be  constructed  so  that  they  are 
independent,  and  costs  and  benefits  so  that  they  are  addi¬ 
tive.  An  experienced  analyst  can  usually  construct  a  model 
to  meet  these  requirements,  at  least  as  a  first-order  appro¬ 
ximation  to  a  more  exact  formulation. 

3.3  Results 


3.3.1  The  structure  of  the  model  -  Table  3-1  presents 
the  design  variables  for  ASH.  Each  of  the  eleven  factors 
can  vary  over  several  levels,  which  constitute  successively 
more  costly  alternatives. 
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Figure  3-1 

COST/BENEFIT  TRADE-OFFS  AND  THE  OPTIMAL  FRONTIER 
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THE  ASH  DESIGN  VARIABLES 


Table  3-2  presents  the  costs  and  benefit  scores 
for  the  alternatives  displayed  in  Table  3-1.  Some  of  the 
cost  estimates  are  absolute  costs  and  others  are  relative 
costs.  The  benefit  scores  represent  the  percent  of  the 
total  benefit  within  a  variable  that  is  obtained  by  a  par¬ 
ticular  improvement  above  the  minimal  or  least  beneficial 
level.  As  such,  the  benefit  6cores  do  not  yet  reflect  the 
relative  importance  of  each  variable. 

One  thing  to  notice  about  the  ASH  design  vari¬ 
ables  is  that  increases  in  cost  do  not  always  correspond  to 
increases  in  benefit.  Four  of  the  design  variables — Communi 
cation  Equipment,  ASE,  Passive  Protection,  and  OEI — show 
this  tendency.  This  feature  results  from  an  effort  to 
provide  levels  that  correspond  to  concrete  options  embodied 
in  the  set  of  available  ASH  alternatives.  Undoubtedly,  it 
is  difficult  to  ascertain  how  much  of  the  total  cost  of  any 
alternative  is  attributable  to  each  factor,  and  this  leads 
to  some  unmatched  orderings  of  cost  and  benefit.  Neverthe¬ 
less,  insofar  as  these  unmatched  orderings  are  accurate, 
certain  design  options,  e.g.,  Level  3  of  Communications 
Equipment,  are  clearly  undesirable. 

Table  3-3  lists  the  weights  assigned  to  the 
variables.  These  weights  reflect  the  relative  benefit  ob¬ 
tained  by  improving  a  variable  from  its  least  beneficial 
level  up  to  its  most  beneficial  level.  In  addition,  the 
weights  reflect  the  importance  of  one  factor  in  relation  to 
the  others.  The  relative  benefit  of  any  particular  level  of 
any  variable  is  obtained  by  multiplying  the  benefit  score  in 
Table  3-2  by  the  appropriate  weight  in  Table  3-3,  then 
dividing  by  100. 

3.3.2  The  efficient  designs  -  The  purpose  of  a  design 
analysis  is  to  discover  the  designs  that  are  efficient, 
which  means  the  designs  that  are  most  beneficial  for  a  given 
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COSTS  AND  BENEFITS  ASSESSED  FOR  THE  ASH  DESIGN  VARIABLES 


Variable 


Weight 


(1)  Communications  6.3% 

(2)  TADS  15.7% 

(3)  Mission  Equipment  Growth  3.1% 

(4)  ASE  7.9% 

(5)  Passive  Protection  11.8% 

(6)  One  Engine  Inoperable  11.8% 

(7)  Crash-Worthiness  9.4% 

(8)  Speed  6.3% 

(9)  VROC  @  4K/95°  11.8% 

(10)  Transmission  Rating  4.7% 

(11)  Navigation  Equipment  11 . 0% 

99.8% 


Note:  The  sum  does  not  equal  100%  due  to  round 
off  errors. 


Table  3-3 

THE  WEIGHTS  FOR  THE  ASH  DESIGN  VARIABLES 
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cost  and  least  costly  for  a  given  amount  of  benefit.  These 
designs  form  a  "frontier"  on  a  cost/benefit  graph  with  the 
inefficient  designs  falling  below  this  frontier.  The  effi¬ 
cient  "frontier"  of  ASH  designs,  based  on  the  assessments  of 
costs  and  benefits  in  Table  3-2,  is  depicted  in  Figure  3-2. 

The  most  striking  characteristic  of  ASH's  effi¬ 
cient  "frontier"  is  that  it  rises  very  rapidly  until  about 
20%  of  the  total  cost  has  been  spent.  Then  it  rises  very 
slowly.  In  addition,  at  20%  of  the  cost,  almost  100%  of  the 
relative  benefit  has  been  obtained.  In  other  words,  certain 
rather  costly  improvements  provide  very  little  additional 
benefit. 


Upon  examining  the  design  that  corresponds  to 
the  kxrk  of  Figure  3-2,  the  picture  becomes  clearer.  A 
design  with  the  most  beneficial  level  on  all  variables 
(except  OEI  and  with  Level  2  on  OEI)  provides  97%  of  the 
relative  benefit  at  19%  of  the  cost.  This  means  that  only 
the  Twin  Engine  Fixed  Rotor  with  SLF  @  4K/95"  is  cost- 
beneficial.  All  other  engine  designs  are  far  too  expensive 
to  be  justified  (see  Table  3-2).  Moreover,  improvements  on 
design  factors  other  than  OEI  should  be  considered  before 
spending  the  money  to  improve  OEI  beyond  Level  2.  (This 
conclusion  is  based,  however,  on  the  estimated  costs  used  in 
the  model.  As  explained  in  Section  3.4,  these  cost  estimates 
have  little  validity  and  are  of  illustrative  value  only.) 

3.3.3  The  proposed  designs  -  Table  3-4  presents  the 
thirteen  proposed  ASH  designs  and  the  levels  they  assume  on 
each  design  variable.  Figure  3-3  depicts  the  cost/benefit 
trade-offs  for  each  of  these  alternative  designs.  These  de¬ 
signs  fall  into  two  groups:  one  group  that  costs  less  than 
20%  of  the  total  cost  and  follows  the  optimization  curve 
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THE  EFFICIENT  "FRONTIER"  OF  ASH  DESIGNS 


COST/BENEFIT  TRADE-OFFS  FOR  THE  13  ASH  CANDIDATES 


DESIGNS  OF  ASH  CANDIDATES 


rather  closely  and  another  group  that  costs  well  over  20%  of 
the  cost  and  falls  short  of  the  optimization  curve.  (Again, 
these  conclusions  are  based  on  invalid  cost  data.) 


The  reasons  for  the  two  groups  becomes  apparent 
when  Table  3-4  is  re-examined.  The  first  group  is  composed 
of  designs  that  have  either  Level  1  or  Level  2  of  OEI,  i.e., 
a  single  or  twin  engine  with  fixed  rotor,  while  the  second 
group  uses  the  more  expensive  engine  designs.  This  indicates 
that  OEI  is  the  driving  force  in  the  present  analysis. 

The  design  analysis  does  not  determine  a  single 
"best"  design.  The  Design  methodology  can,  however,  suggest 
how  pre-specif ied  options  might  be  improved.  For  example, 
in  Design  C,  a  decrease  in  the  capability  of  the  navigation 
equipment  (from  Level  5  to  Level  4)  would  permit  improve¬ 
ments  in: 


(a) 

Communications  (Levels  2- 

4)  ; 

(b) 

Mission  Equipment  Growth 

(Levels  1-3) ; 

(c) 

ASE  (Levels  5-6)  ; 

(d) 

Passive  Protection  (Levels  1-6)  ; 

(e) 

Crash-Worthiness  (Levels 

6-7) ;  and 

(f) 

VROC  (Levels  2-4). 

These  changes  would  improve  the  overall  benefit  of  Design  C 
without  increasing  its  cost.  A  similar  analysis  for  Design  L 
suggests  that  Design  L's  cost  is  so  great  that  it  would  be 
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only  marginally  more  expensive  to  purchase  the  most  bene¬ 
ficial  level  on  all  factors.  This  would  involve  improve¬ 
ments  in: 

(a)  Communications  (Levels  2-4) ; 

(b)  TADS  (Levels  4-6); 

(c)  Mission  Equipment  Growth  (Levels  4-5) ; 

(d)  ASE  (Levels  5-7)  ;  and 

(e)  Passive  Protection  (Levels  4-7) . 

These  suggestions  for  improved  designs  demonstrate  the  type 
of  analysis  for  which  Design  models  are  best  suited. 

3.4  Comments  on  the  Design  Model 

The  Design  model  of  ASH  alternatives  described  above  is 
an  initial  attempt  to  structure  this  problem.  As  such,  the 
model  suffers  from  the  typical  problems  of  initial  models 
which  are  usually  worked  out  in  subsequent  refinements. 

These  include  problems  in  both  the  model's  structure  and 
inputs.  The  results  of  the  model,  therefore,  are  only 
illustrative;  they  are  not  valid  and  serve  no  other  purpose. 
The  discussion  below  details  each  of  these  problems  and  ex¬ 
plains  the  reasons  why  the  model  was  not  refined. 

The  major  problem  with  the  model's  structure  is  the 
interaction  among  the  OEI  variable,  which  addresses  the  type 
of  engine  in  the  helicopter,  and  other  variables.  For  ex¬ 
ample,  the  type  of  engine  determines  the  maximum  permissible 
weight  which  determines  the  amount  of  equipment  that  can  be 
added.  A  possible  way  to  correct  this  problem  would  be  to 
model  the  remaining  variables  within  the  constraint  of  each 
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major  type  of  engine  and  then  to  tie  the  separate  models 
together  in  an  overall  model.  This  solution  is  compatible 
with  the  modeling  approach  but  was  not  undertaken  for  rea¬ 
sons  explained  below. 

The  inputs  to  the  model  were  assessed  in  a  very  short 
period  of  time  using  the  best  available  sources.  There  were 
problems  with  inputs  because  the  time  period  was  so  short 
and  because  the  best  sources  of  data  were  not  always  avail¬ 
able.  The  latter  problem  was  especially  true  for  the  esti¬ 
mates  of  costs.  Costs  often  had  to  be  assessed  by  technical 
experts  who  had  little  confidence  in  their  ability  to  esti¬ 
mate  cost.  In  addition,  some  cost  figures  were  provided  by 
DDI  analysts  in  order  to  get  the  model  up  and  running. 

These  figures  were  intended  to  be  used  for  demonstration 
purposes  only  and  to  be  revised  by  cost  experts.  Such  pro¬ 
blems  with  inputs  are  common  to  all  modeling  efforts  and 
would  have  been  solved  in  the  normal  course  of  refining  the 
model.  However,  since  the  model  was  not  refined,  the  prob- 
blem  remains. 

During  the  process  of  refining  the  Design  model,  we 
discovered  that  the  real  interest  was  in  evaluating  the  13 
identified  ASH  candidates  rather  than  in  devising  the  most 
efficient  ASH  design.  This  is  the  main  reason  for  halting 
work  on  the  Design  model  and  for  concentrating  the  remaining 
effort  on  the  Evaluation  model. 
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4 . 0  CONCLUSION 


In  the  preceding  chapters,  the  nature  of  the  decision- 
analytic  modeling  effort  performed  by  DDI  for  the  ASH  SSG 
has  been  described.  While  the  DDI  modeling  effort  has  not 
recommended  a  single  ASH  candidate  over  all  others,  it  has 
allowed  the  SSG  to  more  fully  understand  trade-offs  of  the 
competing  decision-related  variables  with  one  another.  For 
a  chosen  set  of  importance  weights  allocated  across  these 
variables,  it  is  possible  to  state  which  candidate  is  of  the 
most  worth  to  the  Army  and  which  candidates  are  most  effi¬ 
cient.  All  numerical  assessments  have  been  supported  through 
written  rationale,  and  it  has  been  possible  to  vary  any  score 
or  weight  that  may  be  in  question  and  to  determine  the  im¬ 
pact  of  such  variations  on  the  result. 
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